Collaborative assessments in on-line classrooms

Consider a teacher who needs to assess a large number of assignments. With massive open on-line courses (MOOCs) gaining momentum, it is now common for thousands of students to enrol in the same course, and hence manual assessment by teachers is simply unfeasible. Peer assessments is one way to go when auto-scoring approaches are not possible. Current on-line courses usually use a simple aggregation of peer assessments, but these suffer from two main pitfalls. First, simple aggregation does not take into consideration how reliable each peer assessment is. Second, simple aggregation calculates what the students think of an assignment as opposed to what the teacher (the far more important assessment source) thinks of it. This paper proposes two different models to address these two different pitfalls. These models lay the foundation for future work, where we intend to combine both models into a single one that addresses both pitfalls at once.